There has been considerable skepticism regarding the viability of massive parallelism. The skepticism centers around Amdahl's law, an argument put forth by Gene Amdahl in 1967 ("Validity of the Single Processor Approach to Achieving Large-Scale Capabilities," AFIPS Conference Proceedings 30, 483-485) that for a program with serial work fraction s, the maximum parallel speedup obtainable is bounded by 1/s. This law has led to the assertion that the serial fraction will dominate execution time for any large parallel ensemble of processors, limiting the advantages of the parallel approach.
In the operation of programs, many sequential steps in addition to those in their kernel algorithms are used. These include the steps to obtain parameters, set up the computation, load the program into the machine, gather the result, and create displays. In a parallel system, steps are also included for sending and receiving messages between the processors. The total time of these communications between the processors adds to the communication time of the program. Although kernel algorithms can be sped up proportionately with more processors, prior to the present invention, there has not been a parallel computing system or method in which such programs systems could be used in a manner permitting significant speedup.